Veeam Availability Orchestrator Release Notes 


This document provides last-minute information about Veeam Availability Orchestrator (VAO), including system 
requirements and relevant information on technical support, documentation, online resources and so on. 


The current version of Veeam Availability Orchestrator is available for download at veeam.com/disaster-recovery- 
orchestrator-download.html starting from March 16th, 2021. 


NOTE: 


The release build of Veeam Availability Orchestrator is 4.0.0.2088, while the embedded Veeam Backup & 
Replication build is 11.0.0.837 and the embedded Veeam ONE build is 11.0.0.1379. 


See next: 
e System Requirements 
e Required Permissions 
e Known Issues 
e Technical Documentation References 
e Technical Support 


e Company Contacts 


System Requirements 


Unless otherwise stated, all 3rd party software must be at the latest update or patch level. 
Hardware 


Hardware requirements depend on the size of the managed infrastructure. 


Number of 
Virtualization 100 100-500 500-1000 >1000 
Hosts* 
4 vCPUs (minimum) - 8 vCPUs (minimum) 12 vCPUs (minimum) >16 vCPUs for the 
8 vCPUs - 12 vCPUs - 16 vCPUs VAO server 
(recommended) for the (recommended) for (recommended) for >16 vCPUs for the 
VAO server the VAO server the VAO server Microsoft SOL 
4 vCPUs (minimum) - 8 vCPUs (minimum) 12 vCPUs (minimum) Server database 
8 vCPUs - 12 vCPUs - 16 vCPUs 
(recommended) for the (recommended) for (recommended) for 
Microsoft SQL Server the Microsoft SQL the Microsoft SQL 
database Server database Server database 
Memory 4 GB (minimum) - 8 8 GB (minimum) - 40 GB (minimum) - >70 GB for the VAO 
GB (recommended) for 40 GB 70 GB server 
the VAO server (recommended) for (recommended) for >70 GB for the 
4 GB (minimum) - 8 the VAO server the VAO server Microsoft SQL 
GB (recommended) for 8 GB (minimum) - 40 GB (minimum) - Server database 
the Microsoft SQL 40 GB 70 GB 
Server database (recommended) for (recommended) for 
the Microsoft SQL the Microsoft SQL 
Server database Server database 
Hard Disk 50 GB for product installation and sufficient disk space for Microsoft SQL Server (if installed Locally) 


Space 


* Under the condition that 1 host has 20 VMs deployed. 


Only 64-bit versions of the following operating systems are supported: 
e Microsoft Windows Server 2019 
e Microsoft Windows Server 2016 
e Microsoft Windows Server 2012 R2 
e Microsoft Windows Server 2012 
e Microsoft Windows Server 2008 R2 SP1 


It is not recommended to install VAO on a machine running Microsoft Windows Server Core, or on a Domain 
Controller. 


User Management 


Windows domain-joined machine. 
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Microsoft SOL Server 


Local and remote installations of the following versions of Microsoft SOL Server are supported: 


Microsoft SQL Server 2019 
Microsoft SQL Server 2017 
Microsoft SQL Server 2016 (2016 SP2 Express Edition is included in the setup) 
Microsoft SQL Server 2014 


Microsoft SQL Server 2012 (2012 SP4 Express Edition is included in the setup) 


It is not recommended to use the Express Edition in any production VAO deployments — it should only be used for 
product evaluation. 


Veeam Software 


If a standalone Veeam Backup & Replication server is used for backup and replication task management, it must be 
version 10a or later. 


Additional Software 


All components will be installed during setup. 


For inline Report Template editing, Microsoft Word component of SP2 for Microsoft Office 2010 or later is required. 


Virtualization Platform 


VMware vSphere 6.0, 6.5, 6.7, 7.0 (up to 7.0 Update 1) 


The VAO server must be connected to VMware vCenter Servers. Direct connections to vSphere hosts are not 
supported. 


Storage System 


NetApp ONTAP 9.3, 9.5, 9.6, 9.7, 9.8 


Lenovo DM 9.3, 9.5, 9.6, 9.7, 9.8 
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Required Permissions 
The accounts used for installing and using Veeam Availability Orchestrator must have the following permissions. 


Account Required Permission 


Setup Account The account used for product installation must be a domain user who has the Local Administrator 
permissions on the target machine. 


VAO Service Accounts The accounts used to run VAO services, Veeam Backup & Replication services and Veeam ONE 
services must have Local Administrator permissions on the VAO server. 

The accounts must also be granted the Log on as a service right. For more information on 
Windows security policy settings, see Microsoft Docs. 


VAO Agent Account The account used to install and run a VAO agent on a Veeam Backup & Replication server must 
have both the local Administrator and the Veeam Backup Administrator permissions on the 
server. 

VAO User Accounts The accounts used to log in to the VAO UI must be granted the Allow log on locally right. For 


more information on Windows security policy settings, see Microsoft Docs. 


vCenter Server The account used to connect the vCenter Server to the VAO infrastructure must have 
Permissions administrative permissions. You can either grant the Administrator role to the account or 
configure more granular permissions. For more information, see Veeam Backup & Replication 
Required Permissions and Veeam ONE Required Permissions. 

To be able to open sessions on the vCenter Server system, the account must also have the 
Sessions. Validate session privilege on the root vCenter Server. For more information on session 
privileges, see VMware Docs. 


NetApp Storage The account used to connect the storage system to the VAO infrastructure must be granted 
System Permissions permissions described in section NetApp Data ONTAP Permissions. 


Microsoft SQL Server Different sets of Microsoft SQL permissions are required in the following cases: 

e Installation (remote or local): the current account needs the CREATE ANY DATABASE 
permission on the SQL Server level. After the database is created, this account 
automatically gets a db_owner role and can perform all operations with the database. 

e Operation: the account used to run VAO services, Veeam Backup & Replication services 
and Veeam ONE services requires the db_owner role, as well as permissions to execute 
stored procedures for the configuration databases on the Microsoft SOL Server. 

For more information, see Veeam Backup & Replication Required Permissions and Veeam ONE 
Required Permissions. 


VAO Step Accounts The account used to run the Verify SharePoint step, must be assigned the 
SharePoint_Shell_Access role and must be a member of the WSS_ADMIN_WPG group on the 
processed VM. 

The account used to run the Verify Exchange Mailbox step, must be assigned the 
ApplicationImpersonation role on the processed VM. 


NetApp Data ONTAP Permissions 


Command/ Directory Access/Query Level 
DEFAULT none 
readonly 
all 
network interface readonly 
snapmirror all 
version readonly 
volume all 
vserver readonly 
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Known Issues 


Technical limitations 


VAO does not support failover orchestration for volumes protected by SnapVault replication. If you create and 
try to run a storage plan containing at least one datastore protected by SnapVault replication, the whole plan 
will fail to execute. 


VAO does not support failover orchestration for SVMs protected by SnapMirror SVM replication. Storage 
failover is orchestrated at the volume level only. 


VAO does not support storage failover of VMs with RDM disks. When VAO tries to register such a VM ina 
vCenter Server while executing a storage plan, the server returns the following error: "Unable to enumerate all 
disks." 


Restore plans created in VAO can orchestrate recovery from backup files stored only in repositories of the 
following storage types: direct-attached storage (both Microsoft Windows and Linux), network-attached 
storage (SMB shares) and deduplicating storage appliances (Dell EMC Data Domain, ExaGrid, HPE StoreOnce 
and Quantum DXi). 


If you plan to run DataLab tests using deduplicating storage appliances, note the limitations described in this 
Veeam KB article). 


VAO does not support the following storage types: tape storage and replicating storage infrastructure. 
VAO supports only NSX-T Virtual Distributed Switch (N-VDS) networks v2.4 and later. 


VAO does not support testing of orchestration plans in DataLabs with network mapping from VSS\VDS to N- 
VDS. 


Installation 


During product installation, it is possible to set any custom port numbers except 9402, 8543 and 20443. This is 
because the embedded Veeam Backup & Replication is hardcoded to use those ports. 


When you follow the VAO installation wizard as described in the Veeam Availability Orchestrator User Guide, 
you have an option to use an existing Microsoft SQL Server instance to host the VAO database. When choosing 
the instance, keep in mind that VAO does not support case-sensitive SOL Server databases. 


By default, all VAO Server components are installed all together on a single machine. If you want to install 
Veeam ONE Monitor Client separately on a Windows Desktop OS, run the .MSI file available in the VAO 
installation folder by the following path: <VAO>\monitor\VeeamONE. Monitor. Client.x64.msi. 


If you want to install Veeam ONE Monitor Client separately on a Windows Server OS, simply launch the VAO 
setup wizard, and follow the instructions provided in the Veeam Availability Orchestrator User Guide, section 
Installing Additional Veeam ONE Clients. 


If you try to install VAO version 4.0 to a directory with a non-ASCII name, VAO will not function properly even 
though the installation process will complete successfully. 


Upgrade 


Performance issues on the VAO Server that occur during upgrade may result in issues with VAO agents: 


o The embedded Veeam Backup & Replication server may show an incorrect version. To work around 
the issue, restart the Veeam Availability Orchestrator Service. 


o Remote Veeam Backup & Replication servers may show a timeout issue. To work around the issue, 
repair the affected VAO agents. 


After upgrade, remote users logged in to the VAO UI using the Firefox browser with the default file cache 
settings may experience errors. 
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To work around the issue, reload the page by pressing F5. 


If Veeam Backup & Replication and Veeam Backup Enterprise Manager are installed on the same server, and 
the Veeam Backup Enterprise Manager has been added to VAO, then VAO may lose connection to the Veeam 
Backup & Replication server after upgrade (even though the server will still be shown as ‘Healthy’ in the VAO 
Agents view). To work around the issue, remove and re-add the Veeam Backup Enterprise Manager to VAO. 


If you try to upgrade VAO to version 4.0 while the existing VAO 3.0 databases are protected with SOL 
replication, the upgrade process will fail with an error. This is a known issue related to specific aspects of SQL 
replication. 


If you try to upgrade VAO to version 4.0 while VAO 3.0 is installed to a directory with a non-ASCII name, VAO 
will not function properly anymore even though the upgrade process will complete successfully. 


To work around the issue, reinstall VAO 3.0 to another directory using the existing databases, and then upgrade 
to VAO 4.0. 


VAO UI 


When you try to launch the VAO UI using the Microsoft Edge browser, you get the network error Ox800cO019. 
This is a known Microsoft issue. 


Try reloading the page by pressing Ctrl + F5, or use any other web browser. 


Infrastructure 


If you have a VMware vCloud Director server added to your backup infrastructure, VAO will not be able to 
process VMs managed by vCenter Servers connected to this server. 


To work around the issue, it is recommended to remove the VMware vCloud Director server from the backup 
infrastructure and connect the required vCenter Servers directly. 


To let VAO connect to the Microsoft SQL Server instance that hosts the VAO database, you must specify 
authentication credentials during VAO Server installation. However, if you change the password later, you 
will not be able to modify the provided credentials, and VAO will fail to connect to the SQL Server instance. 


To work around the issue, it is recommended that you use Windows Authentication when choosing credentials 
to connect to the Microsoft SQL Server. Alternatively, Veeam Customer Support can help you resolve the 
issue. 


User roles 


When you perform initial configuration for the VAO server, the Initial Configuration Wizard allows you to add 
users that will be assigned the Administrator role for the server. Please keep in mind that you cannot assign 
the role to a local Administrator. You cannot add local Administrators to any of the user groups provided by 
VAO. Only domain accounts are supported. 


VAO controls access to its functionality with the help of user roles. A user role defines what operations a user 
can perform and what scope of data is available to the user in the VAO UI. The list of users and user groups 
can be edited using the Users and Scopes tab of the Administration section of the VAO UI. However, you 
will not be able to add users or user groups from domains with one-way trust relationships, and VAO will 
display the following error: "The VAO service account cannot access domain <n>. This may be due to 
insufficient domain trusts or security restrictions." 
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Recovery locations 


After you restart the Veeam Availability Orchestrator Service or reboot the VAO Server, storage plans may fail 
to complete successfully and readiness checks may show that the plans are not ready for failover. 


To work around the issue, wait 5-10 minutes for the inventory to be refreshed. 


If network mapping configured for a CDP policy has 2 or more identical target networks, VAO will be unable to 
fail the VM back to a new location. 


When you create a recovery location, you must specify target hosts where recovered VMs will be registered. If 
you add an ESXi host to a recovery location as Compute Resources and then move the host to another 
datacenter, the host will be assigned a new moRef, VAO will consider the host to be another object and the 
configuration of the recovery location will become invalid. As a result, VAO will not be able to use this Location 
for recovery. 


VAO can discover only those vSphere Distributed Switches that are connected to at least one ESXi host added to 
any vCenter Server managed by VAO. Only in this case you will be able to add such a DVS to network resources 
when configuring recovery locations in the VAO UI. 


When VAO restores VMs to a recovery location which has Instant VM Recovery enabled, VMs are added to the 
root resource pool or the root VM folder. 


To work around the issue, move VMs to the required resource pool or folder manually. 


VM groups 


If a VM has its data disk (.vmdk) and configuration file (.vmx) stored on separate datastores, and the VM was 
suspended when storage failover was run, then the datastore where the .vmx file is located must be processed 
by VAO first. If VAO attempts to process the datastore with the data disk first, the VM will fail to power on as 
the .vmx file will not contain the correct datastore GUIDs. 


To work around the issue, edit the .vmx file manually after all storage processing is complete. Contact Veeam 
Support for assistance if required. 


If a VM has a name that starts with a dot, VAO will not be able to process that VM. This is a known VMware 
issue — some functionality is limited for infrastrucutre objects whose names start with a dot. 


If a VM stores its files on a dedicated swap datastore, VAO will exclude the VM from the VM group related to 
that datastore — but only in case the VM is protected by storage replication. If the VM is not protected by 
storage replication, VAO will still include the VM in the VM group. 


Orchestration plans 


Renaming a CDP policy will result in the failure to retrieve the SLA for the policy and also the failure to build 
correct network mapping for the VM. 


Do not rename CDP policies while the related plans are in progress. 


Removing a CDP policy will result in the failure to map IP addresses from the failover range to the new location 
range. T 


Do not remove CDP policies while the related plans are in progress. 


VAO does not support a scenario where you add a vCenter Server to the VAO infrastructure using its IP address 
and to the standalone Veeam Backup & Replication server using its FQDN. In this case, VAO will not be able to 

execute orchestration plans successfully, and the plans will halt with the following error: "Cannot find the host 
on the VBR Server." 


If you connect a vCenter Server to a standalone Veeam Backup & Replication server added to VAO, you must 
also connect the same vCenter Server to the VAO server. Otherwise, VAO will fail to map VM inventory correctly 
and will be unable to locate VM backups and replicas when running orchestration plans. As a result, the plans 
will fail to complete. 
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e Another reason why VAO may fail to map VM inventory correctly is that, by design, Veeam ONE collects 
historical data from the newly connected Veeam Backup & Replication servers for the previous week only. In 
this case VAO will also be unable to locate VM backups and replicas when running orchestration plans. 


To work around the issue, run backup and replication jobs that has not been run for more than one week. 


e If you add new VMs to a VM group, these VMs will not be added to an orchestration plan that contains the VM 
group until the plan enters the VERIFIED or NOT VERIFIED state. This applies to all other stable and active 
states that the plan may acquire. 


For more information on plan states, see the Veeam Availability Orchestrator User Guide, section Working with 
Orchestration Plans. 


e Ifyou have created multiple backup or replication jobs for one VM, and each job has its own target location, you 
will not be able to select which backup or replica to use when running VAO orchestration plans. By design, VAO 
will recover the VM that has the most recent restore point. 


e To protect a VM after failover, you must configure a template backup job on the Veeam Backup & Replication 
server that processes the VM. Otherwise, VAO will not be able to locate the recovered VM, and the Protect VM 
Group step will fail to complete. 


e Ifa plan includes steps that require in-guest scripts to run on virtual machines being recovered, you may get the 
following error: "RPC error:There are no more endpoints available from the endpoint mapper." 


To work around the issue, add the Shutdown Source VM step to the plan. This will power off source VMs during 
the recovery process, and the in-guest scripts will be able to run on VMs successfully. 


e |f you connect a vCenter Server to VAO and then use a different name when configuring the Power VM Actions 
step for a VM in an orchestration plan, VAO will not be able to process the step and will return the following 
error: "VM not found." 


A similar issue may occur if you connect a vCenter Server, add the VM Power Actions step to an orchestration 
plan and specify the name of a host managed by the vCenter Server as the vCenter Name step parameter 
value. In this case, VAO will not be able to process the step. To work around the issue, specify the name of the 
connected vCenter Server as the vCenter Name step parameter value when configuring the step. 


e For datastores connected through the NFSv4.1 protocol, VAO supports failover to a recovery Location only in 
case target hosts included in the location have the NFSv3 export policy enabled (since the recovered datastores 
will be mounted to the hosts through NFSv3). For datastores connected through other protocols, no limitations 


apply. 


e VAO does not support storage failover in a situation where both the target storage system and source storage 
system have the same name, and the source volumes and target volumes have the same name too. In this 
case, VAO will be unable to identify the target volumes, and the plans will halt with errors. 


To work around the issue, rename the target storage system, source storage system, source volumes and 
target volumes. However, when renaming these objects, keep in mind that VAO also does not support storage 
failover in a situation where storage systems have alias names. 


e VAO fails to perform storage failover using a recovery location where the destination LUN and destination 
volume have the same name. 


To work around the issue, rename the destination LUN. 


e If a datastore is backed by several volumes (in a situation where LUNs of the datastore reside on several 
volumes), you will be able to protect such configuration with different SVMs and to create different storage 
recovery locations for these SVMs. However, in this case, VAO will be unable to match the locations for the 
datastore, and the related plans will halt with errors. 


e |f you suspend a VM that stores its disk files on multiple datastores before creating a SnapMirror snapshot, and 
then include this VM in a storage plan, VAO will not be able to execute the plan successfully. The plan will halt 
with the following error: "Unable to access the virtual disk from the host. Either the host is disconnected from 
the datastore or has insufficient privilege." 
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At the moment, VAO does not support orchestration of storage failover for objects of the type Tag and VM 
Folder on standalone Veeam Backup & Replication servers. No matter if you have a backup job with these 
objects configured on a standalone Veeam Backup & Replication server, VAO will still execute and test the 
related storage plans on the embedded Veeam Backup & Replication server. 


Reports 


You may encounter a situation where you connect a vCenter Server to VAO and some of hosts managed by this 
server become unavailable (for example due to a host failure). In this case VAO will not be able to execute the 
Protect VM Group step for VMs that run on the failed hosts. Moreover, the Plan Execution Report will display 
warnings for the Protect VM Group step executed for VMs that run on available hosts. 


When you edit a report template, save it first before closing. If you try closing the file before you save it, MS 
Word will offer you to save changes, but it will lock the file for up to 10 minutes. 


If you have the synchronous protection relationship set up between volumes, VAO may report incorrect RPO for 
these volumes and for all VMs related to the volumes. 


VM console 


It is not possible to use the VM console after the commit failback operation has completed. VAO will attempt to 
open a console for the replica instead of the failed-back VM, and will display the "VM is not powered on" error. 


The issue will be addressed in a future update. 


After you restart the Veeam Availability Orchestrator Service, VAO may fail to open the VM console and may 
display the "Connection Failure" error. 


To work around the issue, close the console and try opening it several minutes Later. 


Plan testing 


When you start on-demand plan testing, configure test scheduling or launch a DataLab, you have an option to 
define the order in which lab groups required to support the test environment will be executed. However, as 
soon as the plan testing process runs, VAO will ignore the specified order and will execute groups in the order 
they have been included in the lab during its creation (or configuration). 


VAO agents 


On the VAO Agents tab of the Administration section of the VAO UI you have an option to Repair a failed VAO 
agent installed on a Veeam Backup & Replication server. However, when you try to do that, you may get the 
following error: "Repair failed -> The specified service has been marked for deletion." 


You will encounter this error in case the VAO agent is locked by another service running on the server. To work around 
the issue, log in to the server and make sure that: 


ilg 


The Microsoft Management Console, Task Manager and Process Explorer are closed. 


If any of these applications is running, close it. 


The Veeam Orchestrator Agent Management Service (VaoAgentSvc), Veeam Orchestrator Agent for Backup 
(VAOBackupAgent), Veeam Orchestrator Agent for Enterprise Manager (VAOEmAgent) have been deleted or 
have the StartUpType value set to Enabled. 


If any of the services has the StartUpType value set to Disabled, restart the server. 


Then try repairing the VAO agent again. 
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Technical Documentation References 


If you have any questions about Veeam Availability Orchestrator, use the following resources: 
e Product webpage: https://www.veeam.com/disaster-recovery-orchestrator.html 
e User guides: https://www.veeam.com/documentation-guides-datasheets.html?prd=vao 


e Community forums: https://forums.veeam.com/ 


Technical Support 
Veeam offers email and phone technical support for customers on maintenance and during the official evaluation 
period. For better experience, please provide the following when contacting Veeam Customer Support: 


e Version information for the product and all infrastructure components 
e Error message and/or accurate description of the problem you are having 


e Log files 


For your convenience, the VAO UI allows you to collect logs for each VAO component separately, or all together. To 
do that: 


1. Switch to the Administration page. 

2. Navigate to Logs. 

3. Select a check box next to the server where the VAO component runs. 
4. Click Download Logs. 


Logs will be saved locally in the default download folder. 


TIP: 


If you want to create an anonymized log file that contains all configuration and statistical information on your 
VAO deployment, download the VAO Infrastructure data log. The log file will be used by VAO product 
management to improve the solution. No information will be shared outside of Veeam at any time. 


To see information added to the log, navigate to the folder used to store files you download and open the .XML 
log file. 


To submit your support ticket or obtain additional information, please visit the Veeam Customer Support Portal. 
Before contacting Veeam Customer Support, consider searching for a resolution on Veeam Community Forums. 


Company Contacts 


For the most up-to-date information about company contacts and office locations, please visit the Veeam Contacts 
Webpage. 
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